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Abstract 

We show that a class of semidefinite programs (SDP) admits a solution that is a positive 
semidefinite matrix of rank at most r, where r is the rank of the matrix involved in the objective 
function of the SDP. The optimization problems of this class are semidefinite packing problems, 
which are the SDP analogs to vector packing problems. Of particular interest is the case in which 
our result guarantees the existence of a solution of rank one: we show that the computation of this 
solution actually reduces to a Second Order Cone Program (SOCP). We point out an application 
in statistics, in the optimal design of experiments. 

Keywords SDP, Semidefinite Packing Problem, rank 1-solution, Low-rank solutions, SOCP, Opti- 
mal Experimental Design, Multiresponse experiments. 

1 Introduction 

In this paper, we study semidefinite packing problems. The latter, which are the semidefinite pro- 
gramming (SDP) analogs to the packing problems in linear programming, can be written as: 

max (C,X) (1) 
s.t. (Mi,X}<bi, 

where C >r 0, and Mi ^ 0, i 6 [I]. The notation X > indicates that X belongs to the set 
of n x n positive semidefinite matrices. Similarly, X y stands for X £ the set of n x n 

positive definite matrices. The space of n x n symmetric matrices §„ is equipped with the inner 
product (A, B) — trace(^4 T -B). We also make use of the standard notation [I] := {1, ...,/}, and we 
use boldface letters to denote vectors. 

Semidefinite packing problems were introduced by Iyengar, Phillips and Stein [8]. They showed 
that these arise in many applications such as relaxations of combinatorial optimization problems or 
maximum variance unfolding, and gave an algorithm to compute approximate solutions, which is 
faster than the commonly used interior point methods. 

Our main result is that when the matrix C is of rank r, Problem (1) has a solution that is of rank 
at most r (Theorem 2). In particular, when r = 1, the optimal SDP variable X can be factorized 
as xx T , and we show that finding x reduces to a Second-Order Cone Program (SOCP) which is 
computationally more tractable than the initial SDP. We present this result and some applications 
in Section 2. Then, we extend our result to a wider class of semidefinite programs (Theorem 4), in 
which not all the constraints are of packing type. The proofs of the theorems are given in Section 4. 
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2 Main result and consequences 



In this section, we state the main result of this article and point out an application to statistics. We 
also discuss the significance of our result for combinatorial optimization problems (the hypothesis on 
the rank of the matrix C appears to be very restrictive). The two theorems of this section are proved 
in Section 4. 

2.1 The main result 

We start with an algebraic characterization of the semidcfinitc packing problems that are feasible 
and bounded. 

Theorem 1. Problem (1) is feasible if and only if every hi is nonnegative. Moreover if Problem (1) is 
feasible, then this problem is bounded if and only if the range of C is included in the range ofj^i Mi- 

The main result of this article follows: 

Theorem 2. We assume that the conditions of Theorem 1 are fulfilled, so that Problem (1) is feasible 
and bounded. 7/rankC = r, then the semidefinite packing problem (1) has a solution that is a matrix 
of rank at most r. In addition, if min rank Mi — r, then every solution of Problem (1) must be of 

i£[l] 

rank at most n — r + r. 

A consequence of our theorem is that when the matrix in the objective function is of rank 1 
(C = cc T ), the computation of the solution X of Problem (1) reduces to the computation of a vector 
x such that X = xx T . The next result shows that this can be done very efficiently by a Second 
Order Cone Program (SOCP). 

Corollary 3. We assume that the conditions of Theorem 1 are fulfilled, and that C — cc T for a 
vector c G R n (i.e. rank(C) = I). Then, Problem (1) reduces to the SOCP: 

max c T x (2) 

s.t. \\A t x\\ 2 < y/b~, i = le[l], 

where the matrices Ai are such that Mi — AfAi. Moreover, if x is any optimal solution of Prob- 
lem (2), then X = xx T is an optimal solution of Problem (1), and the optimal value of (1) is 
{c T xf. 

Proof. The SOCP (2) is simply obtained from (f) by substituting xx T to X and Aj A t to Mj. The 
objective function (C,X) becomes (c T x) 2 , and we can remove the square by noticing that c T x > 
without loss of generality, since if x is optimal, so is — x. □ 

Remark 1. The ratio between the optimal value of Problem (f ) and the value of its best solution of 
rank one has been studied By Nemirovski, Roos, and Terlaky [11]. They show that the value v* of 
the SDP and the value of its best rank-one solution satisfy: 

v* > Vi > , } - ; v* , where u = min(Z,max rank MA. (3) 
~ 1 ~ 21n(2Z/z) ie[i] 

This ratio can be considerably reduced in particular configurations, but to the best of our knowledge, 
the fact that the gap in (3) vanishes when the matrix C in the objective function is of rank 1 is new, 
except in the particular case in which every Mj is of rank 1, too [15]. 
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The present work grew out from an application to networks [4], in which the traffic between any 
two pairs of nodes must be inferred from a set of measurements. This can be modeled by the theory of 
optimal experimental design, which leads to a large SDP. Standard solvers relying on interior points 
methods, like SeDuMi [18], cannot handle problems of this size. However, in a followup work relying 
on the present reduction to an SOCP [17], we solve within seconds the same instances in SeDuMi. 
We next present this application to statistics. 

2.2 Application to the optimal design of experiments 

An interesting application arises in statistics, in the design of optimal experiments (for more details 
on the subject, the reader is referred to Pukelsheim [14]). An experimenter wishes to estimate the 
quantity c T 0, where is an unknown n— dimensional parameter, and c is a vector of n coefficients. 
To this end, she disposes of I available experiments, each one giving a linear measurement of the 
parameter yi = AiO, up to a (centered) measurement noise. If the amount of experimental effort 
spent on the i th experiment is Wi , it is known that the variance of the best linear unbiased estimator 
for c T is c T (^2 i WiMij^c, where Mi = AfAi, and denotes the Moore-Penrose inverse of M. 
The problem of distributing the experimental effort so as to minimize this variance is called the 
"c— optimal problem" , and can be formulated as: 

min c T (S^WiMi)^c (4) 

i 

I 

S.t. Wj = 1. 

i=l 

It is classical to reformulate this problem as a semidefinite program, by using the Schur complement 
lemma and duality theory (see e.g. [15, 16]). The c— optimal SDP already appeared in Pukelsheim 
and Titterington [13], hidden under a more general form: 

max c T Xc (5) 
s.t. (Mi, X) < 1, ie[l], 

x y o. 

In this problem, the design variable w is proportional to the dual variable associated to the con- 
straints (Mi,X) < 1. Note that this is a semidefinite packing problem, in which the matrix defining 
the objective function has rank 1 (C = cc T ). More generally, if we want to estimate simultane- 
ously r linear functions of the parameter £ = (c^9, . . . , cj0), the best unbiased estimator is now a 
r— dimensional vector with covariance matrix 

l 

Cov TO (C) :=if T (^ Wfc M fe )tif, 
fe=i 

where K = [c±, . . . , c r ]. Several criteria can be used for this experimental design problem. Popular 
ones are the A— criterion and the E~ criterion, which aim at minimizing respectively the trace and the 
largest eigenvalue of Cov w (C). These optimization problems can also be formulated as semidefinite 
packing problems. For A— optimality, this packing formulation is given in [16]: 

max c T Xc (6) 

s.t. (Mi,x)<i, 
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where c = [c[ , . . . , c^] T , and Mi is a block-diagonal matrix which contains r times the block Mj on 
its main diagonal. The matrix in the objective function is of rank 1 (C = cc T ), and so Problem (6) 
reduces to a SOCP by Corollary 3. This reduction is of great interest for the computation of optimal 
experimental designs, because SOCP solvers are much more efficient than SDP solvers, and take 
advantage of the sparsity of the matrices A, (whereas the matrices Mi — AfAi used in the original 
SDP formulation (6) are not very sparse in general). 

The E— optimal design SDP is presented in [20] (for the special case in which C = I), and 
takes exactly the form of the scmidcfinitc packing problem (1), with &, = 1 for all i e [I] and 
C = KK T = Y^i=i c i c i T ■ Here, the matrix C has rank r, and so Theorem 2 indicates that the 
E— optimal design SDP has a solution that is a matrix of rank at most r. This suggests the use 
of specialized low rank solvers for this SDP when r is small (cf. Remark 2), which can lead to a 
considerable improvement in terms of computation time. 

Remark 2. We point out that solutions of small rank of semidefinite programs have been extensively 
studied over the past years. Barvinok [3] and Pataki [12] discovered independently that any SDP 
with / constraints has a solution X* whose rank is at most 

V / 8T+T- 1 
" = [ 2 „ ' 

where [-J denotes the integer part. This was one of the motivations of Burer and Monteiro for 
developing the SDPLR solver [5], which searches a solution of the SDP in the form X = RR T , where 
R is a nx r* matrix. The resulting problem is non-convex, and so the augmented Lagrangian algorithm 
proposed in [5] is not guaranteed to converge to a global optimum. However, it performs remarkably 
well in practice, and some conditions which ensure that the returned solution is an optimum of the 
SDP are provided in [6]. Our result shows that for a semidefinite packing problem in which the 
matrix C has rank r, one can force the matrix R to be of size n x r (rather than n x r*), which can 
lead to considerable gains in computation time when r is small. 

2.3 Relation with combinatorial optimization 

SDP relaxations of combinatorial optimization problems have motivated the authors of [8] to study 
semidefinite packing problems. Hence, we discuss the significance of our result for this class of 
problems in this section. 

Semidefinite programs have been used extensively to formulate relaxations of NP-hard combina- 
torial optimization problems after the work of Goemans and Williamson on the approximability of 
MAXCUT [7]. These SDP relaxations often lead to optimal solutions of the related combinatorial 
optimization problems whenever the solution of the SDP is of small rank. As shown by Iyengar et. 
al. [8], SDP relaxations of many combinatorial optimization problems can be cast as semidefinite 
packing programs. Our result therefore identifies a subclass of combinatorial optimization problems 
which are solvable in polynomial time. Unfortunately, this promising statement only helped us to 
identify trivial instances so far. For example, the MAXCUT semidefinite packing problem [8] yields 
an exact solution of the combinatorial problem whenever it has a rank 1 solution. The matrix C in 
the objective function of this SDP is the Laplacian of the graph, and so it is known that 

rank C — N — k, 

where N is the number of vertices and k is the number of connected components in the graph. Our 
result therefore states that if a graph of N vertices has N — 1 connected components, then it defines 
a MAXCUT instance that is solvable in polynomial time. Such graphs actually consist in a pair of 
connected vertices, plus N — 2 isolated vertices, and the related MAXCUT instance is trivial. 

Another limitation for the application of our theorem in this field is that most semidefinite packing 
problems arising in combinatorial optimization (including but not limited to the Lovasz i? function 
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SDP [10] and the related Szegedy number SDP [19], the vector colouring SDP [9], the sparsest cut 
SDP [1] and the sparse principal components analysis SDP [2]) can be written in the form of (1), with 
an additional trace equality constraint trace(X) = 1. In fact, we can show that if such an "equality 
constrained" problem is strictly feasible, then it is equivalent to the following "classical" semidefinite 
packing problem: 

max (C + XI, X)-X (7) 
s.t. (M i: X) < bi, 
traced < 1, 

xto, 

where A is any scalar larger than |A*|, where A* is the optimal Lagrange multiplier associated to the 
constraint trace(AT) = 1 (we omit the proof of this statement which is of secondary importance in this 
article) . Since C + XI is a full rank matrix, our result does not seem to yield any valuable information 
for this class of problems. 

3 Extension to "combined" problems 

The proof of our main result also applies to a wider class of semidefinite programs, which can be 
written as: 

max (C,X) + (R ,Y) +h T X (8) 
s.t. (M t ,X) <b, + (R U Y) +hi T \ ie[i], 

x e S+ Y e S+, A e R q , 

where every matrix Mj and C are positive semidefinite, while the Ri are arbitrary sym- 
metric matrices. The vectors hi are in M 9 . We denote by H the q x I matrix formed by the columns 
hi, . . . , hj. The dual of Problem (9) is: 

min b T (a (9) 

fj,>0 

i 

S.t. C ' 

i=l 

I 

i=l 

h + Hn = 0. 

In the next theorem, we show that if the dual problem is strictly feasible, and the primal is 
feasible, then a solution of rank at most r exists. This theorem is proved in Section 4. 

Theorem 4. We assume Problem (9) is strictly feasible and Problem (8) is feasible, i.e. the following 
two conditions hold: 

(i) 3p > : Ei~PiMi yC, R + ftifc ~< 0, h + HJl = 0, 

(ii) 3(Y,X) e§+xl«: V^ e [I], h + (R, t ,Y) + hi T \ > 0. 
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Then, Problem (8) has a solution (A, Y, A) in which 



In addition, if min rank Mj 

je[i] 

rank A < n — r + r. 



rank X < r :— rank C. 
r, then every solution (X, Y, A) of Problem (8) must be such that 



Remark 3. In the SDP (8), we have introduced an unbounded vector variable A. The reader may 
wonder why, since we could have expressed A as the diagonal of the difference of two positive semidef- 
inite matrices instead. Although subtle, introducing an unbounded variable does make a difference, 
since otherwise the dual strict feasibility assumption (i) would require h + H~p < and h + H~p > 
to hold at the same time (which of course is not possible). 

As in the previous section, we have a result of reduction to a SOCP, which holds when C is of 
rank 1, every Ri = and h = 0. 



Corollary 5. Consider the following "combined" semidefinite packing problem: 

(C,X) 



max 

XGS m , yet 



(10) 



s.t. (Mi,X) < hi A - 

x y o. 



i e 



Assume that C = cc T has rank 1, and 

(i) 3p > : 7Z 2 M, yC, HJL = 0, 

(ii) 3A>0: H T \ + b>0, 

where H = [hi, . . . ,h{\ is the matrix whose columns are the vectors hi 
to the following SOCP: 



max 



T 
C X 



Then, Problem (10) reduces 
(11) 



s.t. 



hi T \ + bi 



<hi X + b t + 1, 



i e 



where the matrices Ai are such that Mi = AjAi. Moreover, if (x, A) is a solution of Problem (11), 
then (xx T ,X) is a solution of Problem (10), and the optimal value of (10) is (c T x) 2 . 

Proof. Theorem 4 guarantees that Problem (10) has a solution (A, A) in which A has rank 1, i.e. 
A = xx T for some vector x. The SOCP (11) is simply obtained from (8) by substituting xx T to A 
and AfAi to Mi. We also used the fact that for any vector z and for any scalar a, the hyperbolic 
constraint 



is equivalent to the second order cone constraint 



2z 
a- 1 



< a 



< a + 1. 



□ 
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Application: c optimal design of experiments with multiple resource constraints 

In a more general setting than the classical c— optimal design problem (4) presented in the previous 
section, w no longer represents the percentage of experimental effort to spend on each experiment, 
but describes some resource allocation to the available experiments, that is subject to multiple linear 
constraints Pw < d, where P is a q x I matrix with nonnegative entries and d is a q x 1 vector. This 
problem arises for example in a network-wide optimal sampling problem [17], where w is the vector 
of the sampling rates of the monitoring devices on all links of the network, and is subject to linear 
constraints that limit the overhead of the routers. We will show that this problem is a "combined" 
semidefmite packing problem which reduces to an SOCP. The resource constrained c— optimal design 
problem reads as follows: 

min c T (Vw 4 M l ) t c (12) 

w>0 z — ' 

i 

s.t. Pw < d. 

We assume that the optimal design problem is feasible, i.e. there exists a vector w > such that 
Pw < d and c is in the range of J2i WiMi. Note that we can assume without loss of generality that 
w > 0. Otherwise, this would mean that the constraints Pw < d, w > force the equality Wi = 
to hold for some coordinate i <G [I], and in this case we could simply remove the experiment i from 
the set of available experiments. 

We can now express the latter problem as an SDP thanks to the Schur complement lemma: 

min t (13) 

tGM, w>0 



S.t. 



c 1 

Pw < d. 



y o. 



Since the optimal t is positive (we exclude the trivial case c = 0), the latter matrix inequality may 
be rewritten as 



T 



by using the Schur complement lemma again. Finally, we make the change of variables fi = tw and 
Problem (13) is equivalent to 

min t (14) 
^>o,t>o 

s.t. li%Mi > cc T 

i=l 

P/j, < td. 

This problem is exactly in the form of Problem (9), for C = cc T , = t, b = [0, . . . , 0, 1] T € 
R l+1 , Mi + i —0, h — 0, H = [P, —d], and for alH e 0, . . . , I + 1, i? l = (we also need to introduce 
a nonnegative slack variable to handle the inequalities as equalities). 

By a similar argument as in the proof of Theorem 2 (see the reduction to Problem (17)), we 
can assume that Ei=i ^ (* ms works because c e Range(Ei Mi)). Therefore, there exists a 
scalar A > such that Mi >- cc T . We set t = min .^ w ^. (t is well defined because w > 0) and 

JI = tw, so that PJI < td, and J2\=i~Pi M i ^ XM * y ccT '• This shows that Problem (14) is 

strictly feasible. In addition, the corresponding primal problem is clearly feasible (for A = 0, since 
b > 0), and thus we can use Corollary 5: the c— optimal design problem with resource constraints (12) 
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reduces to the SOCP (11). We give below this SOCP (with the parameters b, Mi and H defined as 
above), as well as its dual: 



max c T x 

x, X 



P 



} A \ X - 1 



d T X < 1, 
A > 0. 



mm 

ft>0,t>0,a>0,(zi) isra 



< Pi T X + l (Vie[i]) 



i=i 
i 

i=l 

P/x < id 

a.i - fii 



<Cfi+ ^, 



where the vectors pi , . . . ,pi el' are the columns of the matrix P. According to the previous change 
of variable, the optimal design variable w is related to the dual optimal variables n and t by the 
relation w = Moreover, Corollary 5 shows that the optimal value of Problem (12) is the square 

of the optimal value of these SOCPs. 



4 Proofs of the theorems 

Proof of Theorem 1. The fact that the problem is feasible if and only if every bi is nonnegative is 
clear, since X = is always feasible in this case and Mi y 0, X y 0, implies (M i} X) > 0. 

Now, we assume that each bi is nonnegative, and we show that Problem (1) is bounded if and 
only if Range C C Range J^i Mi. The positive semidefiniteness of the matrices Mj implies that there 
exists matrices A t (i e [l]) such that Aj A t = M u and \A\ , • • • , Af ][A{ , • • • , Af } T = £V Mj. We also 
consider a decomposition C = X^fc=i c fe c fc T - F° r an Y factorization M = A T A of a positive scmidefinite 
matrix M, it is known that Range M = Range A, and so the following equivalence relations hold: 

RangeC C Range ^ Mj Vfc € [r], c k e Range(^Mi) = RangeQAf , • • • , Af]) 

<^Vfc e [r], c k e ^QNullspace(^)^ • (15) 

We first assume that the range inclusion condition does not hold. Relation (15) shows that 

3k e [r],3h e E" : Mi e [I], A { h = 0, c fe T /i ^ 0. 

Now, notice that X = ahh T is feasible for all a > 0, since a(A[Ai, hh T ) = < This contradicts 
the fact that Problem (1) is bounded, because (C,X) > a(ck T h) 2 , and a can be chosen arbitrarily 
large. 

Conversely, if the range inclusion holds, we consider the dual of Problem (1): 

min fj, T b (16) 

fj,>0 

s.t. E^MihC 

i 

The range inclusion condition indicates that this problem is feasible, because it implies the exis- 
tence of a scalar A > such that A J2i Mi y C (we point out that a convenient value for A is 
Sfe=i c fc T (Si Mi^Ck] this can be seen with the help of the Schur complement lemma). This means 
that Problem (16) has a finite optimal value OPT < A^^bj, and by weak duality, Problem (1) is 
bounded (its optimal value cannot exceed OPT). □ 
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We next prove that if Theorem 4 holds, then so does Theorem 2. 



Proof of Theorem 2. We will show that under the conditions of Theorem 2, we can assume without 
loss of generality that the dual problem (16) is striclty feasible, so that Theorem 4 applies. More 
precisely, we will see that when the range of C is included in that of J^Mi, we can construct an 
equivalent problem in which the sum of the matrices Mj has full rank, and the conclusion will follow. 
Let us denote by U a matrix whose columns form an orthonormal basis of the range of Mj (the 
matrix U is obtained by taking the eigenvectors corresponding to the nonzero eigenvalues of ^ Mj). 
Note that every matrix Mj can be decomposed as Mj = UM[U T for a given matrix Mj, because 
its range is included in the range of ^ Mj (we have M[ = U T MiU). The same observation holds 
for C, which can be decomposed as C — UC'U T (we have assumed the range inclusion Range C C 
Range J]jM)- Hence, Problem (8) is equivalent to: 

max (C, U T XU) (17) 

s.t. (M[,U T XU) <h, *€[/], 

After the change of variable Z = U T XU (Z is a positive semidefinite matrix if and only if X is), 
we obtain a reduced semidefinite packing problem. The projected matrices in the constraints satisfy 
£\ Mj = U T {Y Jt Mi)U y 0. Therefore, there is a real A > such that A]T\ M t y C, and the dual 
of the reduced problem (17) is striclty feasible, while the primal problem is feasible (since every bi is 
nonnegative) . Hence, Theorem (4) applies, which guarantees the existence of a solution Z of rank at 
most r' = rank C = rank U T CU < rank C = r for Problem (17). Finally, X = UZU T is a solution 
of the initial problem (1) by construction, and rank X < rank Z < r. □ 

Before we give the proof of Theorem (4), we need one additional technical lemma, which shows 
that one can assume without loss of generality that the primal problem is strictly feasible, and that 
the vector space spanned by the vectors h , hi, . . . , hi coincides with the cone generated by the same 
vectors. 

Lemma 6. We assume that the conditions of Theorem 4 are fulfilled. Then, we can construct a 
reduced "combined" semidefinite packing problem 

^rnax^ (C',Z) + (R ,Y) +h T X s.t. Vz e 1, (M[, Z) < b t + (R U Y) + h/ X, 

that has the same optimal value as (8) and that satisfies the following properties: 
(i) 3(Z' y 0,Y' y 0,A') : Vi e I, (Mj, Z') < b t + (Ri,Y') + hj \' ; 
(ii) The cone K generated by the vectors (A&i)j£{o}uz is a vector space. 
(Hi) rank C < rank C; 

(iv) There is a matrix U with orthonormal columns such that if (Z, Y) is a solution of the reduced 
problem, then (X := UZU T , Y) is a solution of Problem (8) (which of course satisfies rank X < 
rank Z). 

Proof. In this lemma, (i) and (ii) are the properties that we will need to prove Theorem 4. Properties 
(Hi) and (iv) ensure that if the theorem holds for the reduced problem, then the result also holds for 
the initial problem (8). We handle separately the cases in which the initial problem does not satisfy 
the property (i) or (ii). If both cases arise simultaneously, we obtain the result of this lemma by 
applying successively the following two reductions. 

Let (X* ,Y* , X*) be an optimal solution of Problem (8) ; the existence of a solution is guaranteed 
by the strict feasibility of the dual problem indeed. We denote by 2o C [/] the subset of indices for 
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which bi + (Ri,Y*) + hi T \* = (note that we have h + (R U Y*) + h^ A* > for all i because 
Mi y implies (Mi,X*) > 0). We define X := [I] \2~ - In Problem (8), we can replace the constraint 
(Mi,X) < b l + (R i ,Y) +hi T \ by (M U X) = for all i £ I , since (AT*,F*, A*) satisfies this stronger 
set of constraints. For a feasible positive semidefinite matrix X, this implies (J2iex Mi, X) — 0, and 
even X^ez = 0. Therefore, AT is of the form UZU T for some positive semidefinite matrix Z, 

where the columns of U form an orthonormal basis of the nullspace of M := X)iei ^» * s obtained 
by taking the eigenvectors corresponding to the vanishing eigenvalues of M ). Hence, Problem (8) is 
equivalent to: 

max (U T CU, Z) + (Rq, Y) + h T X (18) 
s.t. (U T M i U,Z)<b i + (R i ,Y)+h i T X, i£l, 

z±_o, y y o. 

We have thus reduced the problem to one for which 6j + (Ri,Y) + hi T X > for all i, and strict 
feasibility follows (i.e. property (i) holds, consider A' = X*,Y' = Y* + r\\l, and Z' = r\il for 
sufficiently small reals r\\ > and 772 > 0). Moreover, the projected matrix C := U T CU in the 
objective function has a smaller rank than C (i.e. (Hi) holds). Finally, (iv) holds for the reduced 
problem by construction: if (Z, Y, A) is a solution of Problem (18), then (X = UZU T ,Y,X) is a 
solution of Problem (8), both problems have the same optimal value, and of course rank X < rank Z. 

We now handle the second case, in which Property (ii) does not hold for Problem (8). The 
set K = { [ho,H]v, v E M. l+1 ,v > 0} is a closed convex cone. Hence, it is known that it can 
be decomposed as K = L + Q, where L is a vector space and Q C L 1 - is a closed convex pointed 
cone (L = K n (-K) is the lineality space of K). The interior of the dual cone Q* is therefore 
nonempty, i.e. 3A : Vq G Q \ {0}, X T q > 0. Let Ao be the orthogonal projection of A on L 1 - , so that 
Xo T q = X T q > for all q e Q \ {0}, and X T x = for all x e L. Now, we define the set of indices 
1" = {i G [I] : hi e L}, and its complement 2~ = ['] \ ^- For all i € lo, hi = Xi + qi for a vector 
Xi e L and a vector G Q \ {0}, so that Xo T hi = Xq T x^ + Xq T qi = Xq T qi > 0. For the indices 
z G I, it is clear that Xq T hi = 0. Finally, since fao + H ~p = 0, we have — ho G K, so that ho G L and 
h T A = 0. To sum up, we have proved the existence of a vector Ao for which 

Mi e {0} U 2", A T ft,i = and Mi e X , A T ft,i > 0. 

Let (A"*,F*, A*) be an optimal solution of Problem (8). For all positive real t, (X*,Y*,X* + tX ) 
is also a solution, because it is feasible and has the same objective value. Letting t — >• 00, we see 
that the constraints of the problem that are indexed by i £ Iq may be removed without changing the 
optimum. We have thus reduced the problem to one for which (ii) holds. 

□ 

We can now prove the main result of this article (in its extended version for "combined" problems) . 
We will first show that the result holds when each Mj is positive definite, thanks to the complementary 
slackness relation. Then, the general result is obtained by continuity. We point out at the end of 
this section the sketch of an alternative proof of Theorem 2 for the case in which r = 1, based on the 
bidual of Problem (1) and Schur complements, that shows directly that Problem (1) reduces to the 
SOCP (2). 

Proof of Theorem 4- By Lemma 6, we may assume without loss of generality that K = 
conc{ft,o, • • • j hi} D —K and that the primal problem is strictly feasible. The strict feasibility of 
both the primal and the dual problem ensures the existence of solutions for Problems (8) and (9), 
and strong duality holds because the Slater condition is fulfilled. This means that there is no dual- 
ity gap between (8) and (9), and that the pairs of primal and dual solutions ((X* ,Y* , X*), fi*) are 
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characterized by the Karush-Kuhn- Tucker (KKT) conditions: 

Primal Feasibility: Vi G [l], (M U X*) < h + (R t ,Y*) + h/X*, 

x* y o, y* y 0; 

x i 

Dual Feasibility: /x* > 0, ^ ^*M 4 ^ C, #o + ^ ti R i ^ °> ^° + = °5 

i=l »=1 
! ! 

Complementary Slackness: p*M, - C) X* = 0, (i? + ^ M*^) ^* = 0. 

»=i »=i 

Now, we consider the case in which Mi y for all i, and we choose an arbitrary pair of primal 
and dual optimal solutions ((X*,Y*, A*),/z*). The dual feasibility relation implies fi* ^ 0, and so 
X^Mi-^i is a positive definite matrix (we exclude the trivial case C = 0). Since C is of rank r, we 
deduce that 

rank(J^ [i*M l - C) > n - r. 

i 

Finally, the complementary slackness relation indicates that the columns of X* belong to the nullspace 
of — C), which is a vector space of dimension at most n — (n — r) = r, and so we conclude 

that rank A"* < r. 

Note that the same reasoning as above shows that if min ie [i] rank Mj = r, then we have 
rank(^\ /U*-Mj — C) >r — r, and every solution of Problem (8) must be of rank at most n — r + r. 

We now turn to the study of the general case in which M; y 0. To this end, we consider the 
perturbed problems 

max (C, X) + (Rq, Y) + h T X 
s.t. (Mi+eI,X) <fe 4 + (i? 4 ,r)+/ii T A (P e ) 

xyo, y y o, 



and 



min V" Hibi 
>n — ' 



M>0 

4=1 



s.t. ^2tM{Mi+eI)>zC,, (D e 



i=i 

i 

i=i 

h +H(j, = 0. 

where e > 0. Note that the strict feasibility of the unperturbed problems (8) and (9) implies that 
of (P e ) and (D e ) on a neig hborhood e e [0,e ], e > 0. We denote by ((X s , Y e , X e ), a pair 
of primal and dual solutions of (P e )-(D £ ). If e > 0, Mj + £l y and it follows from the pre- 
vious discussion that X s is of rank at most r. We show below that we can choose the optimal 
variables (X s , Y s , X £ , £t £ ) ee ] >eo i within a bounded region, so that we can construct a converging sub- 
sequence (X Sk ,Y Sk , X £h , fi £k )keN, £k — > from these variables. To conclude, we will see that the 
limit (X ,Y , X° , fi°) satisfies the KKT conditions for Problems (8)-(9), and that X is of rank at 
most r. 
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Let us denote the optimal value of Problems (P £ )-{D £ ) by OPT(e). Since the constraints of the 
primal problem becomes tighter when e grows, it is clear that OPT{e) is non-increasing with e, so 
that 

Ve g [0,e ], OPT(e Q ) < OPT{e) < OPT(O). 
Now let e e]0,£o]- By dual strict feasibility assumption (i), there exists a vector JZ > such that 

^■piiMi + el) y ^JI.M, y C, and R + ^/J^ -< 0. (19) 

i i i 

Therefore, we have 

OPT{e) = (C, X s ) + (R ,Y e ) + h T X E < ( + e/), X £ ) + (i? , F e > + h T A e 

< + ( R ^ YS ) + h i TxE ) + ( R o,y s ) + h T \ e 

i 

= Ji T b + (V fcifc + ife, r £ ) + (h + hji) t \ £ , 

^ S v ' 

=0 

where the first inequality follows from (19), and the second one from the feasibility condition (Mi + 
sI,X e ) < hi + (Ri,Y e ) + hi T \ E . The assumption (19) moreover implies that — (X^Mi-^i + -^0) is 
positive definite, so that its smallest eigenvalue A is positive, and 

A trace Y £ < { - (J2~PiRt + Ro),Y £ ) < Ji T b - OPT{e) < Ji T b - OPT(e ). 

i 

This shows that the trace of Y £ is bounded, and so Y £ y is bounded. 
Similarly, to bound X s , we write: 

- C,X £ ) < C£MMi + eI) - C,X £ ) 

i i 

= (J2~Pi(Mi + el), X e ) - OPT(e) + (Ro,Y e ) + h T X s 

i 

< J2M^ + (RuY") + hi T \ e ) - OPT(e) + (Ro,Y e ) + h T X £ 

i 

= Ji T b - OPT(e) + (Y TiiRi + Ro, Y £ ) +(h + Hji) T X s , 



<o 

where the first equality comes from the expression of OPT(e). The matrix J2i~Pi^i — C is positive 
definite and its smallest eigenvalue A' is therefore positive. Hence, 

A' trace X e < JI T b - OPT(e) < JI T b - OPT(e Q ), 

and this shows that the matrix X s y is bounded. 

Now, note that the feasibility of A e implies that the quantity 6, + Y e ) + hi T X s is nonnegative 
for all i £ [I]. Since Y e is bounded, we deduce the existence of a lower bound nij e 1 such that 
h; T A e > to; (Vi e [I]). Similarly, since h T A £ > OPT(e ) - (C,X e ) - (R ,Y £ ), there is a scalar to 
such that h T A e > to . We now use the fact that every vector {—hi) may be written as a positive 
combination of the h k , (fee {0} U [I]), and we obtain that the quantities hj T A £ are also bounded 
from above. Let us denote by H the matrix [h ,H]; we have just proved that the vector HqX £ is 
bounded: 

Set: \\H^X e \\ 2 < to 
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(the latter bound does not depend on e). Note that one may assume without loss of generality 
that X s € Range Hq (otherwise we consider the projection X E p of A e on Range Ho which is also a 
solution since HqX e = HqX £ p . We know from the Courant-Fisher theorem that the smallest positive 
eigenvalue of H Hq satisfies: 

X> (H H T \ - m in vTh o h £ v 
A • (H a H ) — mm ^ . 

Therefore, since we have assumed X e € Range H : 



It remains to show that the dual optimal variables /x £ are bounded. Lemma (6) ensures the 
existence of a matrix Y y and a vector A such that 

v^ € [I], (R l ,Y) +h + hi T X = m > 0. 

By dual feasibility, Rq + J2i k^Ri is a negative semidcfinite matrix, and we have: 

i / 
> ( Ro , Y) + Ml (Ri , Y ) = (R , Y) + lA (Vi ~ h ~ hi T A) . 
»=i i=i 

Hence, we have the following inequalities: 

i 

Vfc G [I], Vkvl < J>/4 ^ b T f i s + X T H fJ , s - (Ro,Y) 



i=i 



= OPT(e) - X h -(R ,Y) 
<OPT(0)-X T h o -(R ,Y), 



and we have shown that fi £ > is bounded. 



We can therefore construct a sequence of pairs of primal and dual optimal solutions 
(X £k ,Y £k , A £fc ,/x efe )fegN that converges, with eu — > 0, eu > 0. The limit X n of this sequence is 

k— >QO 

of rank at most r, because the rank is a lower semicontinuous function and rank X £k < r for all 
k e N. It remains to show that X is a solution of Problem (8). The e— perturbed KKT conditions 
must hold for all k e N, and so they hold for the pair ((X , Y ), X°, by taking the limit (the 
limit of positive semidefmite matrices is a positive semidcfinite matrix because §+ is closed). This 
concludes the proof. □ 

Sketch of an alternative proof of Theorem 2 when r — 1 

When r = 1, there is a vector c such that C = cc T and the dual problem of (1) takes the form: 

min fi T b (20) 

fi>0 

S.t. CC T < k-i^i- 

i 

Now, setting t — £t T b, and w — j, so that the new variable w satisfies w T b = 1, the constraint of 
the previous problem becomes ^ J2 i WiM i . This matrix inequality, together with the fact that 
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the optimal t is positive, can be reformulated thanks to the Schur complement lemma, and (20) is 
equivalent to: 



min t 

ER,ui>0 



S.t. 



(21) 



Ei w i M i 



t 



y o. 



w T b= 1. 



We dualize this SDP once again to obtain the bidual of Program (1) (strong duality holds): 

- P - 2v T c (22) 

s.t. {W,Mi)<0bi, ie[l] 



max 

3eR,zeS + 



Z = 



w 


V 




1 



y o. 



We notice that the last matrix inequality is equivalent toWy vv T , using a Schur complement. Since 
Mi y 0, we can assume that W = vv T without loss of generality, and (22) becomes: 



max 

3eR,uG! 



S.t. 



- (3- 2v T c 

\Aiv\\ 2 < ph, i = le 



(23) 



where Ai is a matrix such that AjAi — Mi. 

We now define the new variables a = \fp, and x = ^, so that (23) becomes: 



max ( max— a 2 — 2ax T c) 

xGR" V a J 

s.t. \\Aix\\ < \fb u i = l£[l}. 



(24) 



The reader can finally verify that the value of the max within parenthesis is (c T x) 2 , and we have 
proved that SDP (1) reduces to SOCP (2). By the way, this guarantees that SDP (1) has a rank-one 
solution. □ 
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